Networks are used to interconnect multiple devices, such as computing devices, and allow the communication of information between the various interconnected devices. The large impact that information technologies have on our daily lives are primarily enabled by the ability of networks to carry data from one place to another almost instantly. Most people today use data transferred through a network in their daily activities, such as using the Internet to get information, communicate (e.g., with cellular phones, e-mail devices, mobile computing devices, or the like), conduct electronic business, and many other daily activities. In the work environment, many organizations rely on networks to communicate information between different individuals, departments, work groups, and geographic locations. In many organizations, a network is an important resource that must operate efficiently. For example, networks are used to communicate electronic mail (e-mail), share information between individuals, and provide access to shared resources, such as printers, servers, and databases, or to collaborate in the preparation of documents. Therefore, a network failure or inefficient operation significantly impacts the ability of enterprises, individuals, or groups to perform their functions.
A typical network contains multiple interconnected devices, including computers, servers, printers, and various other network communication devices such as routers, bridges, switches, and hubs. The multiple devices in a network are interconnected with multiple communication links that allow the various network devices to communicate with one another. If a particular network device or network communication link fails or underperforms, multiple devices, or the entire network, may be affected. To avoid network failures or performance problems network monitoring and management tools are provided to mange the networks.
Network management is the process of managing the various network devices and network communication links to provide the necessary network services to the users of the network. Typical network management systems collect information regarding the operation and performance of the network and analyze the collected information to detect problems in the network. For example, a high network utilization or a, high network response time may indicate that the network (or a particular device or link in the network) is approaching an overloaded condition. In an overloaded condition, network devices may be unable to communicate at a reasonable speed, thereby reducing the usefulness of the network. In this situation, it is important to identify the network problem and the source of the problem quickly and effectively such that the proper network operation can be restored.
One purpose of a network is to provide a forum in which applications can pass information from one location to another across the network. These are commonly referred to as networked applications and are typically designed for specific usage. Examples may include mail applications, financial transactions, streaming media, medical imagery, or airline travel reservations. A given corporate or government network may have dozens or thousands of such applications simultaneously in use.
Timely determination and resolution of network failure and/or performance problems is very important and even critical to the viability of many business enterprises. A network failure can cause very large financial losses. For example, businesses relying on electronic transactions for the sale of products have a critical need for their networks to be operating to enable sales. Even a slowdown of data transfer rates can have a large financial impact due to lower productivity, sales, customer frustration, and the like.
To avoid and quickly resolve network problems, operators are required to monitor and troubleshoot network traffic and correlate application performance with network problems. In highly complex networks, data traffic is transmitted according to a multiple protocols at different levels of the communication process, e.g., open systems interconnection (“OSI”) network layer protocols. Moreover, the communications protocols are also determined by the top level applications, which in some circumstances may be the main source of network delays or failures.
Therefore, network administrators have a need to collect and monitor protocol information in order to understand resolve network performance problems. Conventional network data capturing systems for network monitoring generally require a high level of redundant data storage and computation across the multiple levels of network communications. For example, network traffic measures associated with each of the protocols in each of the levels of a network communication are conventionally collected and stored with respect each level and its associated protocols. This conventional data collection and storage techniques do not scale in terms of performance for the application space where protocol depths are very high due to the large diversity in application level protocols and by being at the top level in the communications infrastructure.
Moreover, some of the conventional protocol identifiers used in the collection of data for network monitoring are highly space inefficient. For example, viz.OID type representations of application protocol lineage (e.g., for hyper-text transfer protocol (“HTTP”) over inter-switch-link protocol (“ISL”) the OID would be “ISL.Ethernet.IPv4.TCP.HTTP”) are very redundant and space consuming. For a typical network communication at least eight levels of protocols in the typical protocol stack are common. In addition, in order to collect network traffic measures for different levels of the protocol stack for a particular communication, the measures are generally collected with respect to each level desired. Thus, for the same communication multiple instances of a flow are stored with associated measures relevant to each of the protocol levels to be monitored. Moreover, there are more than 1500 possible combinations of protocols, each with potentially some relevant significance. For example, communications for an HTTP application through different link layer protocols (e.g., Ethernet.IPv4.TCP.HTTP or HTTP over plain Ethernet versus ISL.Ethernet.IPv4.TCP.HTTP or HTTP over ISL) may result in different traffic performance patterns useful in resolving a network performance problem. Accordingly, the combination of the inefficient storage with the large number of possible combinations result in a required amount of data that is too large for the fast and efficient performance typically required in real-time network monitoring applications.
Accordingly, there is a continuing need for a data collection techniques for network-monitoring systems used in network monitoring and management that can (1) efficiently store and retrieve protocol lineage information, (2) for all levels and combinations of protocols, and (3) with minimalist data collection requirements.